libmem: implement policy-agnostic memory allocation/accounting. #332
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
klihub
force-pushed
the
devel/libmem
branch
2 times, most recently
from
d7744f7 to
6632815
Compare
askervin
reviewed
Jun 19, 2024
There was a problem hiding this comment.
....starting review. I could not yet follow through the whole path if it is or not possible that PreserveContainer requests can return zones different from originals. But if they do... for instance if a single zone of size 32 GB is the only zone that is allowed for two preserved-memory containers, each requesting 32 GB of RAM, then I'd still assume that whoever did this preservation must know what he is doing. If he prefers oom kill of his containers rather than letting them access memory out of the zone, then so be it, we won't touch pinning.
klihub
force-pushed
the
devel/libmem
branch
15 times, most recently
from
8dbbe4f to
9f6da52
Compare
klihub
force-pushed
the
devel/libmem
branch
7 times, most recently
from
0a54715 to
d61ab79
Compare
klihub
force-pushed
the
devel/libmem
branch
4 times, most recently
from
3ed379c to
515f99a
Compare
klihub
force-pushed
the
devel/libmem
branch
from
515f99a to
59732a4
Compare
klihub
force-pushed
the
devel/libmem
branch
from
59732a4 to
0bb2c73
Compare
klihub
force-pushed
the
devel/libmem
branch
2 times, most recently
from
1b6292b to
b03dd81
Compare
klihub
added a commit
to klihub/nri-plugins
that referenced
this pull request
Oct 1, 2024
This will conflict if/when we merge containers#332/libmem. This will need to get dropped, rebased on latest main, the checked and fixed again. So I'll just keep it at the tip until then... Signed-off-by: Krisztian Litkey <[email protected]>
klihub
force-pushed
the
devel/libmem
branch
4 times, most recently
from
7461f05 to
015c985
Compare
Initial implementation of a policy agnostic memory accounting and allocation library. Signed-off-by: Krisztian Litkey <[email protected]>
Signed-off-by: Krisztian Litkey <[email protected]>
Add a brief description of libmem, its basic ideas and core concepts, as package level documentation. Signed-off-by: Krisztian Litkey <[email protected]>
Cut out the original memory accounting and allocation code. Plug in a libmem-based memory allocator instead. Signed-off-by: Krisztian Litkey <[email protected]>
Update unit tests after libmem conversion. Signed-off-by: Krisztian Litkey <[email protected]>
Plug in libmem-based memory allocation (and accounting). Signed-off-by: Krisztian Litkey <[email protected]>
Add support for per balloon type memory configuration and per container overrides using pod annotations. Pass configured or annotated memory types to libmem for allocation. TODO(klihub): per balloon configuration still missing (?) Co-authored-by: Krisztian Litkey <[email protected]> Signed-off-by: Krisztian Litkey <[email protected]>
Add first e2e tests for topology-aware policy memory allocation and type control. Co-authored-by: Krisztian Litkey <[email protected]> Signed-off-by: Krisztian Litkey <[email protected]>
"MEMTYPE=x create ..." in test scripts creates a guaranteed or burstable pod with "memory-type...: x" annotation effective for all containers in the pod. Signed-off-by: Antti Kervinen <[email protected]>
Adds fuzz test generator script, model and runner for topology-aware policy reliability tests on HBM+DRAM+PMEM platform. Signed-off-by: Antti Kervinen <[email protected]>
Yeah, I tend to think so, too. Then we'll need to fix bugs and polish the turd further as we go. I'm just unable to focus enough to get this into any better shape now. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch series
topology-awarepolicy with libmemballoonspolicyFor a more detailed description of the main ideas, core concepts, and some high level
implementation details, see the included package documentation.
TODO items:
ensureNormalMemory()during initial zone selectionRemaining questions related to this initial implementation: